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Abstract 


We  consider  the  nonlinear  filtering  problem  of  a  vector  diffusion  process, 
when  several  noisy  vector  observations  with  possibly  different  dimension  of 
their  range  space  are  available.  At  each  time  any  number  of  these  obser¬ 
vations  (or  sensors)  can  be  utilized  in  the  signal  processing  performed  by 
the  nonlinear  filter.  The  problem  considered  is  the  optimal  selection  of  a 
schedule  of  these  sensors  from  the  available  set,  so  as  to  optimally  estimate 
a  function  of  the  state  at  the  final  time.  Optimality  is  measured  by  a  com¬ 
bined  performance  measure  that  allocates  penalties  for  errors  in  estimation, 
for  switching  between  sensor  schedules  and  for  running  a  sensor.  The  solu¬ 
tion  is  obtained  in  the  form  of  a  system  of  quasi-variational  inequalities  in 
the  space  of  solutions  of  certain  Zakai  equations. 
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1.  Introduction 

1.1.  Motivation  and  preliminaries 

The  problem  of  nonlinear  filtering  of  diffusion  processes  has  received  considerable  atten¬ 
tion  in  recent  years;  see  the  anthologies  [l],  [2],  [3]  for  a  review  of  important  developments. 
In  current  studies  as  well  as  in  related  analyses  of  the  partially  observed  stochastic  control 
problem  with  such  models  [4],  [5],  a  key  role  is  played  by  the  linear  stochastic  partial  differ¬ 
ential  equation  describing  the  evolution  of  the  unnormalized  conditional  probability  measure 
of  the  state  process  given  the  past  of  the  observations,  the  so  called  Zakai  equation. 

A  significant  byproduct  of  these  advances  is  the  feasibility  of  analyzing  complex  signal 
processing  problems,  including  adaptive  and  sensitivity  studies,  in  an  integrated,  systematic 
manner,  without  heuristic  or  adhoc  assumptions.  A  problem  of  interest  in  this  area  is  the 
so  called  sensor  scheduling  problem.  Roughly  speaking  this  problem  is  concerned  with  the 
simultaneous  selection  (according  to  some  performance  measure)  of  a  signal  processing  scheme 
together  with  the  sensors  that  collect  the  data  to  be  processed.  Particular  applications  include 
multiple  sensor  platforms,  distributed  sensor  networks,  large  scale  systems.  For  example,  in 
a  multiple  sensor  platform,  there  is  definite  need  for  coordinating  the  data  obtained  from 
the  various  sensors  which  may  include  radar,  infrared,  sonar,  etc.  The  data  obtained  from 
different  sensors  are  of  varying  quality  and  a  systematic  way  is  needed  for  allocating  confidence 
or  basing  decisions  on  data  collected  from  different  types  of  sensors.  For  example  radar  sensors 
are  more  accurate  than  infrared  sensors  for  long  range  tracking  while  the  opposite  is  true  for 
short  range  tracking.  In  sensor  networks  one  needs  to  coordinate  data  collected  from  a  large 
number  of  sensors  distributed  over  a  large  geographical  area.  Conflicts  should  be  resolved 
and  a  preferred  set  of  sensors  must  be  selected,  over  finite  (short)  time  intervals,  and  utilized 
in  detection,  estimation  or  control  decisions.  Similarly  in  large  scale  systems  there  is  typically 
an  attached  information  network  with  the  objective  of  collecting  data,  processing  them  and 
making  the  results  available  to  the  many  control  agents  for  their  decisions  (actions).  Again 
the  need  for  coordinating  this  information  in  a  systematic  way  is  critical. 

In  such  sensor  scheduling  problems  the  systematic  utilization  of  sensors  should  be  the 
result  of  optimizing  reasonably  defined  performance  measures.  Clearly  these  performance 
measures  shall  include  terms  allocating  penalties  for  errors  in  detection  and/or  estimation. 
But  more  importantly,  they  must  include  terms  for  costs  associated  with  turning  sensors 
on  or  off,  and  for  switching  from  one  sensor  to  another.  Examples  of  such  costs  arising  in 
practice  abound.  Turning  on  a  radar  sensor  increases  the  detectability  of  the  platform  (since 
radars  are  active  sensors)  and  this  should  be  reflected  as  a  switching  cost.  Deciding  to  use 
a  more  accurate,  albeit  more  complex  sensor,  will  require  higher  bandwidth  communications 
and  often  more  computational  power  allocated  to  that  sensor.  In  distributed  sensor  networks 
it  may  mean  the  physical  movement  of  a  sensor  carrying  platform  (such  as  a  helicopter 
or  airplane)  to  a  particular  geographical  location.  In  large  scale  systems  the  utilization  of 
several  (often  hundreds)  sensors  for  decision  making  may  provide  better  average  performance 
but  it  certainly  reduces  the  response  speed  of  the  system  to  changing  conditions,  and  it 
increases  computational  and  communication  costs  both  in  terms  of  hardware  and  software. 
The  latter  arc  obviously  evident  in  large  computer/communication  networks.  These  running 


2 


and  swithching  costs  will  depend  often  on  the  part  of  the  state  space  occupied  by  the  state 
vector,  i.  e.  they  will  be  functions  of  the  state  as  well.  For  example  sensors  have  different 
accuracy  or  noise  characteristics  when  the  state  process  takes  values  in  different  areas  of 
the  state  space.  Also  there  is  cost  associated  with  handling  the  transfer  of  information,  or 
tracking  record,  when  there  are  changes  in  the  set  of  sensors  used;  and  these  costs  often 
depend  on  the  state  process. 

It  is  not  our  intent  to  provide  an  extensive  description  of  applications  here.  Detailed 
descriptions  of  some  of  these  problems  can  be  found  elsewhere;  see  for  example  [6],  [7],  The 
underlying  thread  in  all  these  problem  areas  is  the  existence  of  a  variety  of  sensors,  which  pro¬ 
vide  data  (for  processing)  including  information  of  widely  varying  quality  about  parameters 
or  variables  of  interest,  for  control,  detection,  estimation  etc.  Due  to  the  complexity  of  these 
problems  it  is  important  to  develop  systematic  conceptual,  analytical  and  numerical  methods 
for  their  study  and  to  reduce  reliance  on  ad  hoc,  heuristic  methods  as  much  as  possible.  The 
present  paper  is  offered  as  a  contribution  in  this  direction.  It  provides  a  general  methodology 
to  this  problem  by  reducing  it  to  the  analysis  of  a  system  of  quasi-variational  inequalities  (see 
section  3  for  details).  Numerical  methods  will  be  described  elsewhere  [13]. 

The  sensor  scheduling  problem  is  considered  here  in  the  context  of  non-linear  filtering  of 
diffusion  processes,  and  is  therefore  applicable  to  detection  problems  with  the  same  signal 
models.  Modifications  of  the  results  apply  to  other  situations  including  control.  In  the  next 
section  we  present  a  somewhat  heuristic  definition  of  the  problem,  intended  to  describe  the 
problem  clearly,  at  an  intuitive  level.  The  intricasies  of  establishing  this  model  in  a  rigorous 
mathematical  fashion  are  given  in  section  2,  and  constitute  one  of  the  main  contributions  of 
the  paper. 

1.2.  Preliminary  description  of  the  problem 

The  problem  considered  is  as  follows.  A  signal  (or  state)  process  x(-)  is  given,  modelled 
by  the  diffusion 

dx[t)  =  f[x(t))dt  +  g(x(t))dw(t)  (1.1) 

x(0)  =  £ 

in  JRn.  We  further  consider  M  noisy  observations  of  x(-),  described  by 

dy^t)  =  hi{x{t))dt  +  R^dv^t),  (1.2) 

y’(o)  -  o 

with  values  in  lRd' .  Here  u>(-),  u'(-)  are  independent,  standard,  Wiener  processes  in  JRn,  JRdx 

respectively,  and  Ri  =  Rj  >  0  are  d,  x  d,  matrices.  Further  mathematical  details  on  the 
system  (l.l),  (1.2)  will  be  given  in  section  2.  Let  us  consider  a  finite  time  horizon  \0,T].  To 
formulate  the  problem  of  determining  an  optimal  utilization  schedule  for  the  available  sensors, 
so  as  to  simultaneously  minimize  the  cost  of  errors  in  estimating  a  function  of  x(-)  and  the 
costs  of  using  as  well  as  of  switching  between  various  sensors,  we  need  to  specify  these  costs. 
To  this  end,  let  Ci(x)  denote  the  cost  per  unit  time  when  using  sensor  i,  and  the  state  of  the 
system  is  x;  kX0(x ),  k0I(x)  denote  the  cost  for  turning  off,  respectively  on,  the  ith  sensor  when 
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the  state  of  the  system  is  x.  The  objective  of  the  performed  signal  processing  is  to  compute, 
at  time  T,  an  estimate  4>{T)  of  a  given  function  <f>(x(T))  of  the  state.  Penalties  for  errors  in 
estimation  are  assesed  according  to  the  cost  function 

-  *(r))}  :=  B{|#(i(T))  -  (1.3) 

We  shall  comment  briefly  on  more  general  estimation  problems  in  section  4  of  this  paper. 
In  particular  the  consideration  of  a  quadratric  ce(-)  is  not  a  serious  restriction. 

We  consider  next,  the  set  of  all  possible  sensor  activation  configurations ,  denoted  here  by 
N .  An  element  is  £  fl  is  a  word  of  length  M  from  the  alphabet  {0,1}.  If  the  Ith  position 
is  occupied  by  an  1,  the  ^th  sensor  is  activated  (used),  if  by  a  0  the  tth  sensor  is  off.  There 
are  N  =  2M  elements  in  M .  A  schedule  of  sensors  is  then  a  piecewise  constant  function 
u(-)  :  [0,T]  — ►  M .  We  let  Tj  £  [0,T\  denote  the  instants  of  changing  schedule;  i.  e.  ,  the 
moments  when  at  least  one  sensor  is  turned  on  or  off.  At  such  a  switching  moment,  suppose 
the  schedule  before  is  characterized  by  is  £  M ,  and  after  by  is'  £  M.  Then  the  switching  cost 
associated  with  such  a  scheduling  change  will  be 


k „„'(x)  :=  5Z  ki0(x)  +  koj(x).  (1.4) 

The  total  running  cost,  associated  with  schedule  is  £  h!  will  be 


c*(*)  :=  J2  ci(x)  (1-5) 

(jet'} 

In  (1.4),  (1.5),  the  symbol  (t  £  is}  denotes  the  set  of  all  indices  (from  the  set  {1,2 ,...,M} 
which  are  occupied  by  an  1  in  u  (i.  e.  the  indices  corresponding  to  the  sensors  which  are  on); 
similarly  the  symbol  { *  ^  is}  denotes  the  set  of  indices  corresponding  to  sensors  that  are  off. 

Using  the  above  notation  the  available  observations,  under  sensor  schedule  u(-)  are  de¬ 
scribed  by 

dy(t,u(t ))  :=  h(x(t),u(t))dt  +  r(u(t))dv(t),  (1.6) 

where  it  is  apparent  that  the  available  observations  depend  explicitly  on  the  sensor  schedule 
u(-).  In  (1.6),  for  x  £  JRn ,  is  £  M , 


h(x ,  is)  := 


hHx)XM(i) 


(1.7) 


1  hM(x)X{»}  (M)  J 

a  block  column  vector,  where  in  standard  notation 

1,  if  the  ith  position  in  the  word  is  is  occupied  by  an  1 


Xm(0  := 


0,  otherwise 


(1.8) 


Similarly  for  is  £  ff 
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r(u)  Block  diagonaI{ij?*/2X{„}(t)}, 
where  Ri  are  the  symmetric,  positive  matrices  defined  above.  Finally 

'«‘(0  ■ 
t;(t)  :=  : 

vM{t) 

is  a  higher  dimensional  standard  Wiener  process.  In  view  of  (1.7),  for  all  v  £  N 

h(-,v)  :  mn  -»  1RD , 

while 


(1.9) 


(1.10) 


where 


r(u)  :  1RD  -+  mD, 


D  —  di  +  d?  -f-  •  •  •  +  d\f. 


(1.11) 


(1.12) 


(1.13) 


To  make  the  notation  clearer,  consider  the  case  M  =  2,  N  =  4.  Then  M  =  {00,01, 10, 11} 


and 


h[x,  00) 
h(z,0l) 
h(  x,  10) 
/i(z,ll) 


while 


>•(00) 

>-(10) 

>(01) 

>(11) 


Clearly  the  dimension  of  the  range  space  of  y(-,i/)  is 


’  0 

0 

0 

h2{x) 

= 

hl{x) 

0 

= 

h}{x) 

h2\x) 

‘  0 

0  ‘ 

0 

0 

f  R\'’ 

0 

[0 

0 

'  0 

0 

1 

0 

rT  J 

f  RT 

0 

l  0 

r\ 

y(-,")  is 

(1.14) 


(1.15) 


M 


Du  ■=  d<  xw(0- 

1=1 


(1.16) 
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Of  course  for  all  i/,  y{t,u)  £  JRD . 

Following  established  terminology  (c.f.  [9])  we  see  that  a  sensor  scheduling  strategy  is  de¬ 
fined  by  an  increasing  sequence  of  switching  times  ry  £  [0,T ]  and  the  corresponding  sequence 
/'y  £  M  of  sensor  activation  configurations.  We  shall  denote  such  a  strategy  by  u(-),  where 

u[t)  =  u j,  t  £  [ry,  Tj+i);  j  =  1,2,...  (1.17) 

As  stated  earlier  we  are  interested  in  the  simultaneous  minimization  of  costs  due  to  es¬ 
timation  errors  as  well  as  sensor  scheduling.  We  shall  therefore  consider  joint  estimation 
and  sensor  scheduling  strategies.  Such  a  strategy  consists  of  two  parts:  the  sensor  schedul¬ 
ing  strategy  u  (see  (1.17))  and  the  estimator  <j>.  The  set  of  admissible  strategies  Uad  is  the 
customary  set  of  strategies  adapted  to  the  sequence  of  o-algebras 

«(.)),  s<t).  (1.18) 

That  is,  we  consider  strict  sense  admissible  controls  in  the  sense  of  [4],  For  the  problem 
under  investigation  this  last  statement  must  be  interpreted  very  carefully.  First,  we  have 
indicated  in  (1.18),  that  the  available  past  observation  data  information  a-algebra  depends 
(as  is  evident  from  (1.6)  -  (1.9))  very  strongly  on  the  sensor  schedule  u(-).  This  dependence  is 
non-standard,  as  here  the  dimension  of  the  observation  vector  and  the  noise  covariance  change 
drastically  at  each  switching  time  r,.  In  standard  stochastic  control  formulations  [4],  [5],  the 
dependence  of  y  on  u(-)  is  much  more  implicit.  This  is  a  difficult  part  of  the  formulation 
here,  since  it  prevents  us  from  using  Girsanov  transformations  in  a  straightforward  manner. 
Secondly  (1.18)  means  that  the  switching  times  r,  and  the  variables  u,-,  which  define  u(-), 
must  be  adapted  to  the  filtration  which  depends  essentially  on  the  values  of  r,  and 

v,!  Finally  (1.18)  also  means  that  <f>(T)  must  be  measurable  with  respect  to  We 

shall  describe  a  rigorous  mathematical  construction  of  such  a  model  in  section  2. 

Given  such  a  strategy  the  corresponding  cost  is 

J{u{-)A)  :=  £M*(r))-^(T)|2  (1.19) 

fT 

+  /  c(x(t),u(t))dt  ( 1 .20) 

Jo 

+  52Wx(t)MTi- i)>u(r;))}-  (1-21) 


Here  for 

x  £  JR",  v,  u'  £  M 

c{x,o)  :=  cu{x), 

(1.22) 

(c.f.  Eq. 

(1.5)),  and 

k(x,v,v')  =  k^.(x), 

(1.23) 

(c.f.  Eq. 

(1.4)). 

The  optimal  sensor  scheduling  in  nonlinear  filtering  is  thus  formulated  as  the  determina¬ 
tion  of  a  strategy  achieving 

inf  J(u(-),<j>)  (l-24) 
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among  all  admissible  strategies. 

To  simplify  the  notation  a  little,  let  us  order  the  elements  of  M  according  to  the  numbers 
they  represent  in  binary  form.  For  example  in  the  case  M  =  2,  N  =  4  we  replace  M  = 
{00,01, 10, 11}  by  the  set  of  integers  {l,  2,  3,  4}.  That  is  the  one-one  correspondence  between 
}J  and  {1,2,...,  N }  is  described  by 

v  i — »  (integer  represented  by  u)  +  1  (1.25) 

k  i — »  binary  representation  of  (k  —  l). 

So  in  the  sequel  of  the  paper  we  replace  all  the  v,i /  in  equations  (1.4)  -  (1-23)  by  the 
corresponding  integers  from  {1,  2,  ...,  N  }. 

The  structure  of  the  paper  is  as  follows.  In  section  2  a  precise  mathematical  formulation  is 
given  and  the  corresponding  stochastic  control  problem  is  precisely  defined.  In  section  3  the 
set  of  quasi-variational  inequalities  solving  the  problem  is  derived.  In  section  4  we  offer  some 
comments  and  discussion  for  extensions,  further  developments  and  computational  methods. 


2.  The  Stochastic  Control  Formulation 


2.1.  Setting  of  the  model 

Let  (P.,A,P)  be  a  complete  probability  space,  on  which  a  filtration  Jt  is  given,  A  =  Joo- 
Let  tu(-)  and  z(-)  be  two  independent,  standard  Jr Wiener  processes  with  values  in  IRn  and 
IRD  respectively,  carried  by  this  probability  space.  On  the  same  space  we  consider  also  an 
JRn-valued  random  variable  f,  independent  of  w(-),z(-),  and  with  probability  distribution 
function  7r0. 

We  consider  the  Ito  equation  (1.1),  where  /(•)  is  ^"-valued,  bounded  and  Lipschitz,  while 
g(-)  is  7/2"x"-valued,  bounded  and  Lipschitz.  Letting  a  =  | ggT,  we  assume  a  >  a/n,  where 
a  >  0  and  /„  is  the  n  x  n  identity  matrix.  The  Lipschitz  property  is  unnecessary  and  can  be 
easily  removed  using  Girsanov’s  transformation  (i.e.  consider  weak  solutions  of  (1.1))  [8].  It 
is  assumed  here  to  simplify  the  technicalities  not  related  with  the  main  issues  of  the  paper. 
Under  these  assumptions  (l.l)  has  a  strong  solution  with  well  known  properties  |8j.  Note 
that  under  P,  a(-)  is  independent  of  x(-). 

Consider  next  functions  h'(-),  i  =  1  ,...,M,  from  lRn  into  Md' ,  which  are  bounded  and 
Holder  continuous.  We  shall  denote  by  L  the  infinitesimal  generator  of  the  Markov  process 

*(•)  n  2 

L  :=  aiAx)o^; + (2J) 


or  in  divergence  form 


where 


*:=  £ 


dx{ 


,  ,  a  "  a 


(2.1a) 


ai{x)  :=  -/,(x)  + 

3  =  i 


da,Ax) 

dxj 


(2-1  b) 
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Let  us  next  consider  an  impulsive  control  defined  as  follows.  There  is  a  sequence  Ti  < 
T2  •  •  •  <  Tk  <  •  •  •  of  increasing  ^-stopping  times.  To  each  time  r,-  we  attach  an  ^.-measurable 
random  variable  u,-  with  values  in  the  set  of  integers  {  1,  2,  . ..,  N}  *.  We  define 

u{t)  =  Ui,  ^  <t  <  r,+1,  *  =  0, 1,2, . . .  (2.2) 

and  set  r0  =  0.  We  require  that 

Ti  |  T  as  i  t  co,  (2-3) 

while  Tk  =  T  is  possible  for  some  finite  k. 

Let  Ui  be  the  element  of  corresponding  to  u,  via  (1.25). 

Then  define 

h(x,u(t))  :=  h(x,Ui),  r,-  <  t  <  ri+1,  (2.4) 

where  h(x,v)  is  defined  by  (1.7),  in  terms  of  the  given  functions  h‘(-).  Clearly  h(-,u(t))  maps 
JRn  into  JRD  for  all  sensor  schedules  u(-)  and  is  obviously  bounded  and  Holder  continuous  in 
x.  Define  also 

r(u(t ))  :=  r(i/,),  r,  <  t  <  Ti+1,  (2.5) 

where  r(-)  is  defined  by  (1.9),  in  terms  of  the  given  matrices  72,,  i  =  1,2 ,...,M.  Clearly 
r{u[t ))  maps  J72°  into  JRD  for  all  sensor  schedules  u(-)  but  it  is  singular.  Next  we  define 
h(x,u)  to  be  the  vector  valued  function 

R;l/2h'(x)XM(i) 

h{x,u)  :=  R7ll2h<(x)X{„}(i)  (2.6) 

.  R'Mll2h"{x)X{„}(M) 

with  Xm(0  defined  as  in  (1.8).  Let 

h(x,u(t ))  :=  h(x,Vi),  t{  <  t  <  r,+1.  (2.7) 

Clearly  h(-,  u{t))  maps  IRn  into  1RD  for  all  sensor  schedules  u(-)  and  is  obviously  bounded 
and  Holder  continuous  in  x.  We  shall  refer  to  u(-)  as  the  impulsive  control.  As  we  shall  see,  it 
describes  essentially  the  decision  to  select  at  a  sequence  of  decision  times  one  of  the  functions 
h(-,k),  k  £  {1, 2, . . . ,  N}.  This  is  the  precise  mathematical  implementation  of  the  sensor 
selection  decision  described  in  the  introduction. 

To  see  that  indeed  this  is  the  case,  we  can,  with  the  above  preparation,  use  Girsanov’s 
measure  transformation  method.  Let  us  then  consider  the  process 

C(0  =exP {Jq  h{x(s),u{s))Tdz(s)  IIM*(s)>“(s))I|2<M  (2.8) 

1  Recall  that  N  =  2M  and  the  binary  representation  of  each  integer  1,  2,  N  determines  a  sensor  activation 
configuration  by  (1.25). 
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where  T  denotes  transpose,  ||  •  ||  is  the  JR°  norm.  Note  that  the  process  u(t )  is  adapted  to  Tt. 
Then  since  x(-)  is  adapted  to  7tw  C  7t  and  u(-)  is  cadlag  [8],  (2.8)  is  well  defined.  Moreover 
since  h  is  bounded,  by  Girsanov’s  theorem  [8],  [14],  f(-)  is  an  ^-martingale.  We  can  thus 
define  a  change  of  probability  measure 

*  =  m 

and  consider  the  process 

v(t)  —  z(t)  -  j  h(x(s),  u(s))ds.  (2.10) 

By  Girsanov’s  theorem  [8],  [14],  under  the  probability  measure  Pu(')  on  v(-)  is  a 

standard  ^-Wiener  process  with  values  in  IRD.  Furthermore,  by  the  independence  of  tu(-) 
and  z(-),  w(-)  remains  a  standard  2Rn-v alued,  J<-Wiener  process  which  is  independent  of 

u(-).  Finally  £  remains  independent  of  tv(-),  v(-)  while  keeping  its  probability  law,  denoted 
by  ttq .  Thus  x(-)  also  retains  its  probability  law  under  PU('K 

To  relate  this  construction,  i.e.  (2.2)  -  (2.10)  with  the  M  noisy  observations  (sensors) 
loosely  described  in  the  introduction  (c.f.  in  particular  eq.  (1.6)),  observe  that  (2.10)  can  be 
written  as 


~dP~ 


(2.9) 


r{u{t))dz{t)  =  h(x[t),u{t))dt  +  r(u{t))dv{t)  (2.11) 

in  view  of  (1.7),  (1.9),  (2.4),  (2.5),  (2.6)  and  (2.7).  Indeed 


'  x'/’xmW 

0  0 

Ri  1/2/i1(x)x{^,}(1) 

r(u(t))h(x,(u(t)))  = 

0 

R^l,2h2[x)x{ui){  2) 

; 

0 

RMl2hM(x)x{v,){M) 

=  h(x,Vi),  Ti<t<Ti+1.  (2.12) 


To  give  a  precise  meaning  to  (1.2),  or  (1.6),  let  us  introduce  the  process 

y(t,u(t))  :=  y"'(0,  U  <  t  <  r1+1  (2.13) 

where 

dyl’i(t)  r{ui)dz{t )  =  h{x(t),Ui)  dt  +  r(//t)  dv(t).  (2.14) 

It  is  clear  that  if  we  select  u(t)  —  u,  Vt,  where  v  has  0  everywhere  except  for  one  1  in  the 
ith  location,  then  (1.2)  results.  It  is  also  rather  plain  that  yv{t )  G  BlDl’  and  that  in  this  case 
the  Wiener  process  r(u)v(-)  is  also  H^-dimensional  (see  (1.16)  for  the  definition  of  D „).  The 
process  yu'(t)  represents  exactly  the  observation  which  is  available  in  [r,,  r,+1). 
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The  next  issue  that  we  wish  to  clarify  relates  to  the  measurability  question  that  we 
discussed  in  section  1.2,  after  eq.  (1.18).  For  any  u(-),  given  the  construction  of  y(*, «(•)), 
above  we  can  now  consider  as  defined  by  (1.18).  We  shall  say  that  u(-)  is  admissible , 

denoted  u  £  Uaj,  if  u(f)  is  J^'^ b  measurable,  t  >  0,  where  J^'u^  is  constructed  as  above. 
This  more  precisely  means  that  the  Tj  are  ^-stopping  times  or  that 

(r,-  <  t}  C  7tv(’u(»  (2.15) 

and  that 

Vi  £  ITf’u  f  ».  (2.16) 

Note  that  since  7^  ,u ^  ^  C  Tt  for  any  sensor  schedule  u(-)  adapted  to  jf  Ti  are  “('))_ 

stopping  times  they  are  also  ^-stopping  times,  and  the  above  construction  (2.8)  -  (2.14)  is 
still  valid.  The  implication  of  (2.15),  (2.16)  is  that  one  should  check  that  an  optimizing 
strategy,  obtained  by  some  procedure,  must  satisfy  the  admissibility  conditions.  Clearly  Uad  is 
nonempty  as  strategies  u(t)  —  v,  t  £  [0,T],  obviously  are  admissible.  Also  strategies  with 
fixed  switchings  are  also  admissible.  Note  that  for  an  admissible  control  q  Jf . 

We  have  thus  established  in  this  section  the  precise  mathematical  models  of  nonlinear 
filtering  problems  where  selection  of  sensors  is  possible.  In  particular  we  have  succeeded 
in  circumventing  the  subtleties  associated  with  the  definition  of  admissible  sensor  schedules 
discussed  in  section  1.2.(2) 

2.2.  The  optimization  problem 

For  the  dynamical  system  described  in  2.1,  we  consider  now  the  cost  functional  (1.19) 
where  the  underlying  probability  measure  is  PuO.  As  indicated  in  the  introduction,  the 
general  problem  where  the  function  <f>  will  be  in  a  nice  class,  e.g.,  bounded  C2,  or  polynomial, 
or  C°°  can  be  treated  along  identical  lines.  To  simplify  the  notation  we  have  chosen  to 
formulate  the  problem  for  4>(x)  =  x.  The  technical  difficulties  for  this  case  are  identical  to 
the  ones  in  the  more  general  cases  discussed  above,  particularly  since  this  is  unbounded 
on  JRn.  For  this  choice  the  selection  of  the  optimal  estimator  4>{T)  is  the  conditional  mean 

4>{T)  =  Eu^{x{T)  |  /rv(  'u(  ))},  (2.17) 

where  Eu^  denotes  expectation  with  respect  to  Pu^>.  Let  p[u,t)  denote  the  conditional 
probability  measure  of  x(<),  given  7^  on  JRn .  It  is  convenient  to  express  (2.17)  as  a 
vector  valued  functional  of  p(u,t) 

4>{T)  -  <Km(u,T))  =  [  xdfi(u,T).  (2.18) 

J  Rn 

We  shall  further  assume  that  the  running  and  switching  cost  functions  c, (•), k,y (•), 
i,  j  €  {l,...,7V},  introduced  in  (1.4)  and  (1.5)  have  the  following  regularity 

c,' (•) ,  k,y  (•)  are  in  Cb[Rn)  (i.  e.  bounded  and  continuous)  (2.19) 


2Sincc  r(u(t))  is  a  singular  matrix,  this  stage  is  more  delicate  than  in  standard  stochastic  control  theory,  where 
7f  would  suffice. 
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As  a  result  of  this  simple  transformation  we  can  rewrite  the  cost  as  a  function  of  the 
impulsive  control  u(-)  only  (i.e.  the  selection  of  ^(-)  has  been  eliminated): 

./(«(•))  =  E“H{MT) -#(«(«,  T))f  +  fT  c(z(t),u(t))dt 

Jo 

oo 

+  J2  Kx{Ti)i  «(ri))xr,<T},  (2.20) 

J=1 

where  Xr,<T  >s  the  characteristic  function  of  the  fl-set  {to;  r,(u; )  <  T}.  We  further  assume 
that  the  switching  costs  are  uniformly  bounded  below 

k(x,i,j)>k0,  x  €  IRn,  t,  j  €  {1,. . .  ,N}  (2.21) 

with  ko  a  positive  constant.  Note  that  as  a  consequence  of  (2.20)  if  for  some  admissible  it(-) 
with  positive  probability,  the  number  of  times  rt-  <  T  is  infinite,  then  the  cost  J(tz(-))  will 
be  infinite.  Therefore  for  T  finite  the  optimal  policy  will  exhibit  a  finite  number  of  sensor 
switchings. 

The  optimal  sensor  selection  problem  can  now  be  stated  precisely  as  the  optimization 
problem 


P  :  Find  an  admissible  impulsive  control  u*(-)  such  that 


inf  ./«)), 
v(-)euad 


•>  9-7  1 


where  Uaj  are  all  impulsive  control  strategies  adapted  to  or  equivalently  satisfying 

(2.15),  (2.16).  Problem  P  is  a  non-standard  stochastic  control  problem  of  a  partially  observed 
diffusion. 

2.3.  The  equivalent  fully  observed  problem 

In  this  section  we  transform  the  problem  of  section  2.2,  to  a  fully  observed  stochastic 
control  problem,  by  introducing  appropriate  Zakai  equations.  As  is  customary  in  the  theory 
of  nonlinear  filtering  jl],  [2],  [3],  [4],  let  us  introduce  the  operator 

pM  ),1)W  =  £{?(<)#*(<))  I  (2.23) 

for  each  impulsive  control  u(-).  The  notation  is  chosen  so  as  to  emphasize  the  dependence  on 
u(-),  which  is  due  to  the  dependence  of  f(-)  on  u(-)  as  introduced  in  eq.  (2. 8). 3  The  operator 
(2.23)  maps  the  set  of  Borel  bounded  functions  on  Hln,  into  the  set  of  real  valued  stochastic 
processes  adapted  to  Note  that  p(u(-),t)  can  be  viewed  as  a  positive  finite  measure 

on  Eln.  It  is  the  unnormalized  conditional  probability  measure  of  x(t)  given  [2], 


3But  tlie  expectation  is  with  respect  to  P  and  not  PU^K 
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With  the  help  of  these  measures  we  can  rewrite  the  various  cost  terms  in  (2.20)  as  follows: 


=  E{p(u(-),T)(0)}, 

(2.24) 

where 

«(x)-=lli  ?(“(')’  r)M|i! 

l,X  p(u(.),T)(l)l!  ’ 

(2.25) 

with  x  representing  the  function  x(x)  •=  x  and  1  the  function  l(x) 
straightforward  computation  implies  that 

i,  x  e  mn.  a 

£“(){IMr)  -  *W«.T))f}  =  E{Vr(p(u(-),T))} 

(2.26) 

where  is  the  functional  on  finite  measures  on  lRn  defined  by 

=  m(x2)  - 

(2.27) 

where  X2{x)  —  ||x||2,  x  £  JRn ,  and  n  is  any  finite  measure  on  Mn  such  that  the  quantities 
p(x2)  and  fi(x)  make  sense. 

Next 

Eu^{[  c(x(t),u(t))dt }  =  E{$(T)  [  c(x(t),  u(t))dt} 

Jo  Jo 

=  E{J* 

=  E{J* E{s(T)  \r,}c(x(t),u(t))dt} 

-  E{f°  s(t)c(x(t),u(t))dt},  (2.28) 

because  x(t),u(t )  are  measurable  with  respect  to  Jt  and  £■(•)  is  an  ^-martingale.  Now  define 
a  map  C  with  values  in  Cj(iR”)  via 

C(ui)  :=  cu>.('),  «,  6  {1,2,...,  TV}.  (2.29) 

Then  in  view  of  (2.29),  (2.23),  we  can  rewrite  (2.28)  as 

E<1{J*  c(x(t),  u{t))dt)  =  E{j*  E{{(t)c(x(t),  u(t))l^’u<>}dt} 

-  E{J°  p(u(-),t)(C(u(t))dt}.  (2.30) 

Finally 

}{k(z(rt),  u^,-!),  u(r,))xr,<r}  =  «fo_i),  tt(n))Xr,<r} 

=  E{E{$(Ti)k(x(Ti) ,  u(Ti-i)  ,u(Ti))xTi<T  |^(  ,U(  )}} 

=  £{?(«(•)>  n)(^(ti(7-,-i),u(r,)))xT.<r}-  (2.31) 
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Here  we  have  introduced  the  function  K  with  values  in  Cj(JRn),  via 

K(ui,xij)  —  u,-,  uj  G  {l, 2, . . . ,  N},  (2.32) 

and  we  utilized  the  admissibility  of  u(-).  Note  that  in  the  simpler  case  where  ct(-),k,y(-), 
t,  /  6  {1, 2, ,  N}  are  constant  independent  of  x,  (2.30)  simplifies  to 

e<){Jq  c(x(t),u(t))dt}  =  E{fo  p(u(),t)(X)cu{t]dt}  (2.33) 

and  (2.31)  simplifies  to 

^u(){A:(x(r,),u(rt_1),u(r<))xr.<T}  =  -&{ku._„u.Xr<<rP(«(),  r, )(!)}.  (2.34) 

Utilizing  (2.26),  (2.30),  (2.31)  we  can  rewrite  the  cost  corresponding  to  policy  u(-),  given 
in  (2.20),  as  follows 

•'(*(•))  =  E{*(p(u(),T))  +  £  p(u(),  t)(C{u(t)))dt 

oo 

+  (2.35) 

«=i 

In  (2.35)  we  have  succeeded  in  displaying  the  cost  as  a  functional  of  the  unnormalized  con¬ 
ditional  measure  p(u(-),-)  which  is  the  “information”  state  of  the  equivalent  fully  observed 
stochastic  control  problem.  To  complete  this  transformation  we  need  to  derive  the  evolution 
equation  for  p(u(-),-),  i.e.  the  Zakai  equation.  We  turn  into  this  problem  next  and  derive  a 
weak  form  of  the  Zakai  equation  for  p(u(-),-)  in  the  following  lemma.  Here  C l'1  denotes  the 
space  of  all  functions  t P(x,t)  on  lRn  x  JR  which  are  bounded,  continuous  together  with  their 
first  and  second  derivatives  with  respect  to  x,  and  first  derivatives  with  respect  to  t. 

Lemma  2.1:  For  any  xp  €  Cb‘  we  have  the  relation 

/>(«(•), t)(0(t))  =  n0(jj(0))  +  f  p(u{-),s){~-  +  L£)ds 

Jo  os 

+  /  (2.36) 

Jo  «=i 

where 

/7,(u(s))^>]  (x)  :=  h,(x,u(s))^(x),  i  =  1,2, . . . ,  D,  <p  G  C62 

^(s)(x)  :=  x!j{x,s),  (2.37) 

and  h{  is  the  ith  component  of  h  (see  (2.6)). 

Proof: 

Let  /?(•)  G  L°°(0,T;  JRD)  given  and  consider  the  ^-martingale  p(t),  defined  by 

dp(t)  =  p{t)(3{t)T  dz{t),  p{  0)^1. 


(2.38) 
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Recall  that  by  definition  of  f(t)  (c.f.  eq.  (2.8)) 

d${t)  =  ${t)h{x{t),u{t))T  dz{t),  C(0)  =  1.  (2.39) 

Therefore  by  Ito’s  rule  [8] 

<*(f(0/»(0)  =  + P[*))T  &*{*) 

+  hT(x{t),u(t))(3{t)dt\ 

<T(0)p(0)  =  1,  (2.40) 

2  j 

and  since  ip  €  Cb ' 

drp(x(t),t)  =  +  Lrp(x(t),t))dt 

+  [sjrp(x(t),t)\T  g(x(t))dw(t),  (2.41) 

where  L  is  given  in  (2.1).  Therefore  suppressing  some  arguments  for  ease  of  notation 

d  [xp(x{t),t)  f(t)p(t)l  =  c(t)p(t)l(~-  +  Lxp  +  hT  pip)dt 

+  S/^T gdw{t) -\- xp(h  + /3)T dz{t)\.  (2.42) 

In  (2.41),  (2.42)  we  used  the  notation  S/xp  =  , . . . ,  j~~)T ■  Integrating  (2.42),  and  taking 

expectations  we  deduce 

E{xp{x{t),t)$(t)p{t)}  =  7r0(^(0))  +  E{f°  £(s)p(<s){^j  +  Lip  +  hTPxp]ds}.  (2.43) 

We  can  then  write 

e{  jo  <{s]p[s)[ih + Lrp]ds}  =  e{  j:  e^(s^ + l w  i  &a')]w 

=  E{Jq  p(s)p(u(-)>s)(.^;  +  Lxp)ds} 

=  E{p{t)  j^p{xi{-),s){~  +  Lxp)ds}  (2.44) 

by  virtue  of  the  7i-martingale  property  of  p(-).  Similarly 

E{fo  C[^)p(s)h(x{s),u(s))TP(s)xp(x(s),s)ds} 

=  E{p{t)  f  f(s)^(*(s),s)/i(x(s),u(s))7dz(s)} 

Jo 

=  E{p{t)  f  52p{u{-),s){hi{-,u{s))xP(-,s))dzi{s)},  (2-45) 

Jo  i-i 

where  in  the  first  equality  we  have  used  the  representation  p(t)  =  1  +  /d/,(5)0(a)T<k:(-s)»  and 
the  well  known  insomorphism  between  Ito  stochastic  integrals  and  L 2  [8j.  Finally 

E{ip{x{t),t)${t)p{t)}  =  E{p{t)p{u(-),t){j)[t))}. 


(2.46) 
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Using  (2.44),  (2.45),  (2.46)  in  (2.43)  we  obtain 

E{p{t){p{u{-),t){j)[t))  -  ttq ( 0 (0) )  -  f  p{u{-),s)(^-  +  Lr/j)ds 

Jo  os 

~  /  5Zp(tt(*)»*)(^i(«(*))^(*))^(«)]}  =  0. 

*  0  • _ i 


(2.47) 


t  =  l 


We  can  replace  in  (2.47)  p(t)  by  a  linear  combination  of  such  variables,  with  different  (3 . 
The  set  of  corresponding  variables  is  dense  in  L2  (fl ,  7Z ,  P) .  However,  the  random  variable  in 
the  brackets  in  the  right  hand  side  of  (2.47)  is  clearly  in  L2(Ct,7^'u^\P)  and  therefore  in 
L2(ft ,  7*,  P)  since  7^  ^  C  7tz.  Then  (2.47)  implies  the  result  of  the  lemma  (2.36). 

As  a  remark  we  would  like  to  note  that  the  assumed  nondegeneracy  of  x(-),  implies  that 
the  solution  of  (2.36)  is  unique.  This  can  be  proved  in  general  under  our  working  hypotheses, 
for  solutions  which  are  measure-valued  processes.  Here  we  outline  such  a  proof  for  the  case 
when  these  conditional  measures  are  absolutely  continuous  with  respect  to  Lebesgue  measure 
on  Mn;  i.e.,  in  the  case  unnormalized  conditional  densities  exist.  For  this  we  need  to  assume 
in  addition  that 


7 r0  has  a  density  p0  with  respect  to  Lebesgue  measure ;  po  6  L2{IRn) 
Let  us  denote  by  L*  the  formal  adjoint  of  L  (see  (2.1),  (2.1a),  (2.1b)): 


r  *  A  d  ,  .  d  "3 

*,J  =  1  1  J  *  — 1  * 

and  consider  the  Hilbert  space  form  of  the  Zakai  equation  [10] 


dp  =  L*  pdt  +  ph(-,u(t))T dz(t) 
P(  0)  =  Po- 


(2.48) 


(2.49) 


(2.50) 


The  function  space  in  which  the  solution  is  sought  is 

L2(0,  A,  P;  C (0,  T-  L2(JRn)))  n  (0,  T\  H1(JRn))  (2.51) 

Here  II1  is  the  usual  Sobolev  space  on  Rn  [l  1]  and  the  subindex  ))  in  the  second  L 2 

space,  denotes  that  the  solution  is  adapted  to  the  filtration  7^  t  >  0.  It  follows  from 

the  results  of  E.  Pardoux  [llj,  that  there  exists  a  unique  solution  of  (2.49)  in  the  function 
space  (2.50),  under  the  assumptions  made  here.  We  can  then  establish  the  following. 
Lemma  2.2:  The  following  property  holds 


p(u(-),t)(ip)  =  ( p(u(-),t),rp ),  (2.52) 

Vip  in  L2(JRn)  and  bounded,  where  (•,•)  denotes  inner  product  in  L2(JRn). 

Proof: 

By  slight  abuse  of  notation  we  use  the  same  symbol  to  denote  the  conditional  unormalized 
measure  and  density  (whenever  the  latter  exists).  Let  us  prove  inductively  that 
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p{u(-),Ti  V  (t  A  ri+l))(ip)  -  (p(u(-),r,  v(i  A  r1+1)),V>),  (2.53) 

where  the  left  hand  side  notation  refers  to  the  measure  appearing  in  (2.36),  while  the  right 
hand  side  notation  to  the  solution  of  (2.50),  which  is  uniquely  defined.  Suppose  then  that 
(2.53)  holds  for  i-1,  and  therefore  in  particular 

P(u(-)>r>)(^)  =  {p{u[-),Ti),ip),Vip.  (2.54) 


Consider  now  the  solution  tj  of 

C\ 

+  Lt]  =  -ph(-,u(.s))r/?(s),  s  6  (r,,ry  V  (t  A  r,+1)) 

r}(x,Ti  V  (t  A  rl+i))  =  ip(x)  (2.55) 

where  ip  £  C^(IRn)  and  (3  is  a  smooth  deterministic  function  with  values  in  IRD .  From  the 
assumptions  on  f,  g  and  h'  (it  is  here  that  we  use  the  assumed  Holder  continuity  of  h'),  we 
can  assert  that  the  solution  of  (2.55)  belongs  to  C42,1(iRn  x  (r,,  ry  V  (t  A  r1+1))),  for  any  sample 
u>,  (l  1  ] .  Therefore  (2.36)  implies  (using  (2.55)) 


P(u(')>r*  v  (t  A  ri+1))(ip)  = 


+ 


P(u(-),ri)(v(ri)) 

/r,V(/ Art+1)  & 

Y  P(u()>  s)(Hi  iu{s))v(s))Pj(s)ds 

‘  i=i 

[  (  +  ^P(u(-),'S)(Hry(u(s))r?(s))daJ(s), 

Jr'  y= l 


(2.56) 


where  Hj  is  as  defined  in  lemma  2.1,  and  rj(s)(x)  :=  r](x,s).  Therefore  by  Ito’s  rule 
p(u(-),Ti  V  (t  A  rt  fl))(^)p(rt  V  (t  A  r<+1))  =  p(u(-),  ri)(^(r,))p(rl) 

f 

rr,v(«Ar,  +  1) 


+ 


jC 


T,v(iAr,+1) 

+ 1  [p(5)X^p(u(0>fHtfy(u(5))p(5)) 

j=i 

D 


P(S)  Y  ?(“(')>  5)  (P  («))&  (s)]^;  (<*)• 

J  =  1 


(2.57) 


Hence 


F;{p(u(-),r,  V  (<  A  Ti+1))(ip)p(Ti  V  (t  A  r, •+,))} 

=  ^{p(«(-),r,)(^(r,))p(r,)>.  (2.58) 


On  the  other  hand  from  (2.50)  and  (2.55)  we  obtain 


(p(u(-),r,  v  (<  A  r, ■+,)),  V>)  =  (p(u(-),r,),^(r,)) 

rr,v(£Ar1+l) 

^(p(u(-),  s)/l;  (•,  u(s)),  ^(s))d2j  (s) 

y=i 

f  r,V(tArt+1)  ^ 

XKp(u(-)»5)1  //;(«(5))p(s)/?y(s)ds,  (2.59) 
y=i 


+  r 

JTi 

-r 
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and  thus  also 

#{(p(u(0>r»  v  (*  A  n+i )),V>Mr.  V  (t  A  r,+1>  =  ^{(p(ti(-),r<),^(Ti))p(Tv}.  (2.60) 

But  from  the  inductive  hypothesis  (2.54),  the  right  hand  sides  of  (2.58)  and  (2.60)  are  equal. 
Hence  the  left  hand  sides  coincide.  Varying  /?,  we  easily  deduce  that  (2.53)  holds,  at  least  for 
if)  €  C£°(JRn),  which  is  sufficient  to  conclude  the  proof  of  the  lemma. 

With  this  result  we  can  rewrite  the  cost  (2.35)  as  follows 

J(u(-))  =  E{9(p(u(-),T))+  fT(P(u(-),t),C(u(t)))dt 

Jo 

oo 

+  Y  Xr,<r(p(«0>r.)>#(«.-i> «»))}  (2.61) 

«= i 

where  (see  (2.27)) 

*(P(«(0. r»  =  (P(u(-),T),X2)  -  (2-62) 

Since  the  expression  (2.62)  involves  unbounded  functions  we  have  to  show  that  it  makes  sense. 

At  this  point  it  is  useful  to  introduce  a  weighted  Hilbert  space  in  order  to  express 
\fr(p(u(-),T))  in  a  more  convenient  form.  To  this  end  let 

p{x)  =  1+  ||  x  ||4  (2.63) 

and  L2(Mn;p)  denotes  the  space  of  functions  ip  such  that  <pp  G  L2{JRn).  Define  in  a  similar 
way  the  space  L1(JRn\p).  From  the  discussion  of  existence  and  uniqueness  of  solutions  of 
(2.50)  in  the  functional  space  (2.51)  and  if 

PoeL2{lRn;p)nL1{mn',p), 

it  is  easy  to  check  that  (2.50),  under  the  assumptions  made  in  section  2.1,  has  a  unique 
solution  in  the  space 

L2( f),  A ,  P;  C( 0,  T;  L2(JRn;  p)  n  Ll{JRn-,  p)))  n  L2(0,  T;  Hl{IRn\ p))  (2.64) 

where  Hl{Mn\p)  is  the  obvious  modification  of  H1(lRn).  This  justifies  that  the  quantities 
arising  in  (2.62)  have  a  meaning. 

We  note  that  J(u(-))  is  indexed  implicitly  (we  do  not  include  this  in  our  notation)  by 
7 r0  (or  p0 )  and  u(0)  =  j,  j  £  {1  which  is  determistic  since  it  is  -measurable,  by 

construction. 

We  close  this  section  by  rewriting  the  dynamics  (2.50),  in  terms  of  the  originally  given 
observation  nonlinearities  h\  and  with  forcing  inputs  the  processes  y’(-)  introduced  in  (2.13), 
(2.14).  In  view  of  (2.5),  (2.6),  (2.7),  (2.13),  (2.14)  we  have 

M 

h(-,u{t))Tdz{t )  =  Yi  h}T{-)X{ui}  (y)^71/2^(t),r,  <  t  <  r,+i 
j=i 
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(where  we  have  written  z  =  (21, 22,  •  •  •  zm]T) 

=  E  hjT(-)  X{u{){j)RJlR)l2X{Vi}U)dzj{t),  Ti<t<  Ti+i 
J=1 


=  6(-,Vi)Tdy(t,i/i),  Ti  <t  <  r,+i 

=:  6(-,u(t))Tdy(t,u(t )), 


where 


Ri'h'Wxwi  1) 


[  R-JhM{x)X{l/}{M)  J 

Therefore  the  system  dynamics  (2.50)  can  be  written  equivalently 

dp{u{-),t)  =  L*p{u(-),t)dt  +  p{u(-),t)S(-,u{t))Tdy(t,u{-)) 

p(u(-),0)  =  po, 


(2.65) 


(2.66) 


where  y(t,u(t ))  is  defined  in  (2.13),  (2.14).  This  makes  precise  the  construction  of  a  Zakai 
equation  driven  by  “controlled”  observations  alluded  to  in  the  introduction.  It  also  becomes 
now  clear  that  the  spaces  described  by  (2.51),  (2.64)  are  the  appropriate  ones  as  far  as 
solutions  of  (2.50)  or  (2.66)  are  concerned. 


3.  The  solution  of  the  optimization  problem 

3.1.  Setting  up  a  system  of  quasi- variational  inequalities 

Let  us  consider  the  Banach  space  H  =  L2(lRn;p.)  D  L1[JRn\p)  and  the  metric  space  H  + 
of  positive  elements  of  H.  Let 

B  :=  space  of  Bore!  measurable,  bounded  functions  on H+ 

C  :=  space  of  uniformly  continuous,  bounded  functions  on//+.  (3.1) 

Let  us  now  define  semigroups  <&y(£)  on  S  or  C  as  follows.  Consider  (2.50)  with  fixed  sched¬ 
ule  u(i)  =  /,  and  let  py  denote  the  corresponding  density  p(-,j).  Then  for  j  <E  (1,2,. . . ,  N} 

dpj  =  L*pj  dt  +  pjh 7  dz(t),  py (0)  =  7 r,  (3.2) 

where 

h?  :=  (3.3) 


We  set 


^;{0(^)(7r)  =  R{R(Pj,*(0)}>  F  e  8  or  C, 


(3.4) 
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where  pJiT  indicates  the  solution  of  (3.2)  with  initial  value  7T.  It  is  easy  to  see  that  is  a 
semigroup  since  Pj(t)  is  a  Markov  process  with  values  in  H+ .  It  is  also  useful  to  introduce 
the  subspaces  Bi  and  C\  of  functions  such  that 


|F(7T)| 


Ill’ll  1  S11P  ,-v  ||  f  <  00 
7TG//+  1  +  II  K\ 


(3.5) 


where  ||  tt  ||m=||  tt  ||x,i(r-;m).  The  spaces  B\  and  C\  are  also  Banach  spaces.  They  are 
needed,  because  we  shall  encounter  functionals  with  linear  growth  in  the  cost  function  (2.61). 
To  simplify  the  statement  and  analysis  of  the  quasi-variational  inequalities  that  solve  the 
optimization  problem  considered  here,  we  give  the  details  for  the  case  N  —2  only  in  the 
sequel.  We  shall  insert  remarks  to  indicate  how  the  results  should  be  modified  for  the  general 
case.  Let  us  introduce  the  notation 


(for  example) 

Ci(7r)  =  (Ci,7T) 

where  a  slight  abuse  of  notation,  in  denoting  the  funct 
symbol,  has  been  allowed.  Similarly  the  functional  on  H+ 

IIOcx)!!2 


C, 

:=  C(i>),  i 

Ki 

:=  K(  1,2) 

k2 

:=  «•(  2,1). 

.  functions,  one  can 

.,  in 

denoting  the  f 

(3.6) 


(3.7) 


*00  =  (tt,x  )  - 


(tt,!) 


belongs  to  C\  since  it  is  positive  and 

*00  <  Kx2)  <  IMU 


(3.8) 


(3.9) 


Consider  now  the  set  of  functionals  £/i(7r,t),  U2(n,t)  such  that 


U  u 

,Ut 

e 

C(0,T]  Ci) 

>  0, 

U2{;t)>0 

T) 

= 

II 

* 

t/i(* 

■>o 

< 

*i(s  -  t)Ui(n,s)  +  J 

t 

-  t)Ci  (7r)<fA 

U2[  n 

,0 

< 

$2(s  -  t)U2(n,s)  +  J 

r’<z>2{\- 

t 

-  t)C2(n)d\ 

Vs 

> 

t 

Ui{* 

■*o 

< 

tfi00  +  £/2(M) 

U2  (vr 

,0 

< 

KiW  +  lhfat). 

In  the  sequel  we  will  occasionally  use  the  notation  Ui(s){-n)  =  t/i(7r,s),  i  —  1,2. 


(3.10) 
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3.2.  Existence  of  a  maximum  element 


We  shall  refer  to  (3.10)  as  the  system  of  quasi-variational  inequalities  (QVI).  Our  first 
objective  is  to  prove  the  following. 

Theorem  3.1.  We  assume  that  the  conditions  on  the  data  f,g,h *  introduced  in  section  2.1 
hold.  Then  the  set  of  functionals  U\,U2  satisfying  (3.10)  is  non-empty  and  has  a  maximum 
element,  in  the  sense  that  ifU\,U2  denotes  this  maximum  element  and  U\,U2  satisfies  (3.10), 
then 

Ui  >  UUU2  >  U2. 

The  proof  will  be  carried  out  in  several  steps.  In  fact  there  is  some  difficulty  due  to  the 
functional  ^(n).  We  shall  modify  it  in  order  to  assume  that 

0<*(tt)  <*(*,!)  (3.11) 

where  is  a  constant.  We  shall  prove  the  theorem  with  the  additional  assumption  (3.11), 
prove  the  probabilistic  interpretation,  i.e.  the  connection  with  the  infimum  of  (2.61).  The 
probabilistic  formula  will  be  next  used  in  an  approximation  procedure.  We  can  approximate 
for  instance  the  functional  'F  defined  by  (3.8)  in  the  following  way.  Set 


"INI2 

1  +  Ml! 

n 


dx  — 


11/ 


_ 

(1+M2)»/i  " 

/  7T  dx 


which  clearly  satisfies  (3.11)  with  =  n. 


(3.12) 


Proof  of  Theorem  3.1  under  the  assumption  (3.11).  The  set  of  functionals  satisfying  (3.10) 
is  a  subset  of  Bi  or  Cj  defined  in  (3.5).  However  for  this  subset  the  norm  (3.5)  is  unnecessarily 
restrictive.  For  those  functionals  it  is  sufficient  to  set 


If  =  L2{fRn)nL1{IRn) 

H+  =  set  of  positive  elements  of  H  (3.13) 

and  to  consider  Bi,  C i  the  space  of  Borel  or  continuous  functionals  on  H+  such  that 

IfMI 


imii  =  sup 


<  oo 


We  shall  then  study  the  system  (3.10)  with  Ci  replaced  by  C i.  Let  us  notice  that 

H+  c  H+ 


(3.14) 


and  if  one  considers  a  functional  F  in  Si  or  Cj,  its  restriction  to  H+  belongs  to  Si  or  C i;  the 
injection 


F  —*■  restriction  of  F  to  H+ 

is  continuous  from  Si  or  C\  to  Si  or  Ci.  Therefore  replacing  in  (3.10)  C1hyC1  gives  a  stronger 
result. 
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i„J!rhfafl\lhe  Pr°°f,““U  to  Writc  the  “y-nbol  ~  and  write  ft.C,  instead  of  B,,C,  H* 
instead  of  H  ,  the  norm  ||  ||i  is  then  given  by  (3.14). 

caseThe  Pr>VS(thien  a"  adaptation  of  lhe  methods  of  Bensoussan-Lions  [9]  to  the  present 
case  m  order  to  take  mto  account  the  fact  that  we  use  C,  instead  of  C 
First  note  that 

ll*i(*)!kci:c1,  <  1  (3  15i 

where  £(CU  C,)  is  the  space  of  linear  continuous  operators  from  C,  into  itself.  Indeed  we  have 

JiiMfMi  = 

i  +  ta,  1)  1  +  (n,l) 

<  [|  Pl[  t1  +-g(Pi,.(0,ll)) 

1 +  (*,!) 

=  Ill'll. 


since  from  (3.2) 


Therefore 


£(Pi, *(*)>!)  = 
ll$i(0(^)l|i  <  HFjl! 


(3.16) 

(3.17) 


which  implies  (3.15).  1  ’ 

Note  also  that  a  solution  of  (3.10)  will  satisfy 

CMM)  <  *,(r  -  t)Ut(n,T)  +j\l{\  -  i)c,MdA  (3.18) 

and  due  to  positivity,  we  also  have 

IlcMOIIi  <  11^1(^)111  +  IIC'iK^r-t)  <  ijCjjKr- 1)  (3 19i 

where  ||C'1||  =  supr(7j(x). 

As  it  is  customary  in  the  study  of  Q  VI  we  begin  with  the  corresponding  obstacle  problem, 

Uuu2  e  c^TjCi) 

Ui(;t)  >  0,U2(-,t)>0 

U\  (tt,  1 )  =  U2[tt,T)  =  ^(7r) 

Ul(*,t)  <  $l(5-0^l(7T,s)  $i(A  -t)Cy{Tl)dX 

U2(7T,t)  <  $2(s- t)U2(7r,s)  +  J'  $2(A  -  i)C2(7r)dA 

Vs  >  t 

Ui{n,t)  <  Ki{n)  +  $2(n,t) 

U2{tt,1)  <  K2(n)  +  (3_20j 


where  we  assume  that 


fi,ft  €  c(o,r,c,) 

Cl(7T,t)  >  0,  >  0 

Cl(^,T)^2(7T,T)  >  ^(tt). 


(3.21) 
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We  then  have  the  following. 

Proposition  3.1:  For  as  in  (3.21)  the  set  of  UuU2  satisfying  (3.20)  is  not  empty  and 
has  a  maximum  element. 

It  is  clear  that  for  given,  the  system  of  inequalities  (3.20)  can  be  decoupled  and 
UUU2  can  be  considered  separately.  Let  us  then  omit  indices  momentarily  and  consider 


u 

e 

C(Q,T-  Ci) 

U(;t) 

> 

0 

U{7t,T ) 

— 

s)  +  1’  $(\-t)C(n)d\ 

U(ir,t) 

< 

$(s-t)U(jr, 

Vs 

> 

t 

U(n,t) 

< 

where  f  stands  for  instance,  for  -fi  To  prove  proposition  3.1,  it  suffices  to  show 

that  (3.22)  has  a  maximum  element.  This  can  be  done  by  the  penalty  method.  So  we  look 
for  Uc  solving 


£/,(() 

=  $(s-i)t/£(s)  + 

^  *(A  -  t)[C{ n)  -  -(Ue( A)  -  <T(A))  +  ]rfA 

J  t  6 

for 

t  <s<T 

U<(T)(  k) 

=  ^(7r) 

uc 

e  C(0,T;C1) 

Ue(;t ) 

IV 

o 

We  can  then  assert 

Lemma  3.1  There  is  a  unique  solution  of  (3.23). 
Proof:  Notice  that  (3.23)  is  equivalent  to 


V'{t)  =  *(T  -  t)Ut(T)  +  [T  S(A  -  <)(<?(*)  -  -  c(A))  + jdA  (3.24) 

J  t  (. 

and  also  to 

U{(t)  =  -t)^(w)  +  J*  -  t) 

\C( tt)  +  |c/e(A)  -  Vf(A)  -  f(A))+]dA  (3.25) 

Let  us  define  the  transformation  Tc  of  C(0,  T;C i)  into  itself  using  the  right  hand  side  of  (3.25). 
Then  the  latter  can  be  written  as  a  fixed  point  equation 


U(  =  T(U(  (3.26) 

Using  (3.11)  and  (3.15)  one  can  show  precisely  as  in  Bensoussan-Lions  [9,  p.488]  that  some 
power  of  Tc  is  a  contraction.  Hence  the  result  of  the  lemma  follows. 
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One  then  can  also  prove  as  in  (9,  pp.489  -  490],  that  if  e  <  <  K ,  then  0  <  U(  <  Ut>. 

As  in  [9,  pp.494  -  495]  one  then  shows  that  as  e  J.  0,  Ut  j  U  which  is  the  maximum  element  of 
(3.22).  The  convergence  takes  place  in  <7(0,7’;  Ci).  This  establishes  Proposition  3.1. 

We  can  then  proceed  with  the 

Proof  of  Th eorem  3.1:  (Continuat ion ) 

Let  us  consider  the  map  H  mapping  C(0,T;Ci)  *  C(Q,T\C i)  into  itself  defined  by 

■ff(fi.ft)  =  {UUU2)  (3.27) 

where  the  right  hand  side  represents  the  maximum  element  of  (3.20).  Let  now 


U?(7r,t)  =  *t(T-t)9(*)  +  J  ^1(X-t)Cl(n)d\ 

(3.28) 

Consider  <T,  (<),  &■(£), 

i  =  1,2  such  that 

(3.29) 

and 

6(0“  ft  (0  [0,1]. 

(3.30) 

Then  we  have 

0  <  Htfu  6)  ~  H(s x.ft)  <  7(1  -7Wfi,6) 

(3.31) 

where 

i  ^  k0 

7  “  k0  +  V  +max(]jC1)|,||C,2)|)T 

(3.32) 

Indeed,  setting 

K  =  1  -  7(1  -  7') 

(3.33) 

we  have  to  prove  that 

<ff(?i,fc). 

(3.34) 

Let  us  set 

(C/x,C/2)  =  J7(fl,f2) 

(t>r,l>2)  =  #(6,6)- 

(3.35) 

We  need  then  to  show  that 

^  /C C/" 2  ^  f/2. 

(3.36) 

If  we  can  establish  that 

KiTx(a-)  + /c6(tt,<)  <  /fl(7T)  +  f2(7T,<) 

«-Ka(*)  -f  K£i(7T,t)  <  -Ka(*)  + ft  (*,*)> 

(3.37) 

23 


then  (3.36)  is  implied  by  the  monotonicity  properties  of  Variational  Inequalities.  But 


6  k,  0(1  -7)  < 

hence  it  is  enough  to  establish  that 

kK^tt)  +  K£2(n,t)  <  Kx{%)  +  (1  -  7)  6k, 0 

/c/Gk) +  «6k»0  <  K2(n)  +  (1  ~7)6k>0 

The  first  of  (3.39)  will  be  satisfied  if 

k  -  (1  -  7)]6(M)  <  (1  -  *0#ik) 

or  if 


(3.38) 


(3.39) 


7*6 (»r,<)  <  (1  -  7,)i^i(7r)- 


(3.40) 

(3.41) 


But  observe  that 


6k, 0  <  ttfk.O  <  (#  +  ||<72||r)  k,i) 

So  it  is  enough  to  choose  7'  so  that 

V(*  +  ||C2||T)k,l)  <  (1  -  V)*b(jr,  1) 


(3.42) 


where  Ar0  is  the  uniform  lower  bound  (2.21),  since  Ki(ir)  >  Ar0k,  1).  This  last  inequality 
requires 


7'  < 


ko  +  *  +  \\C2\\T 

In  an  identical  fashion,  the  second  of  (3.39)  will  be  satisfied  if 

k0 


7 


'  < 


k0  +  'f  +  ||Ci||r' 


(3.43) 


(3.44) 


So  both  of  (3.39)  will  be  satisfied  if  we  choose  7'  according  to  (3.32).  The  proof  of  the  theorem 
then  proceeds  via  the  standard  iteration 


(ur^ur1)  =  h(u[\u?) 


(3.45) 


as  in  [9,  pp.512  -  514  . 


Remark:  The  extension  of  this  result  to  the  general  case  TV  yt  2  is  straightforward.  The 
system  (3.10)  has  TV  functionals  t/j, . . . ,  Un-  Everything  in  (3.10)  is  the  same  except  for  the 
last  two  inequalities  which  are  replaced  by 


min  (Kij(n)  +  Uj(n,t)),  1 

jV* 


1,...,TV 


(3.46) 


One  again  introduces  the  system  (3.20)  where  the  last  two  inequalities  are  replaced  by 
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<  min  [Ki  j{n)  +  fy(«r,0)»  (3.47) 

where  &  G  C(0,T;Ci),  and  satisfy  the  remainder  of  (3.21).  One  then  establishes  the  analog 
of  Proposition  3.1  by  penalization.  The  analog  of  Theorem  3.1  is  established  by  introducing 
a  map  II  mapping  C(0,  T\Ci)n  into  itself  defined  by 

HUute,  -  -  • ,  f/v)  —  {Uu  U2  ■ . . ,  Us) 

where  the  right  hand  side  is  the  maximum  element  of  the  analog  of  (3.20). 

3.3.  Existence  of  an  admissible  sensor  schedule 

Our  objective  in  this  section  is  to  show  that  the  maximum  element  U\,  U2  of  the  QVI  (3.10) 
provides  the  value  function  for  the  optimization  problem  (2.61),  (2.66)  when  the  assumption 
(3.11)  holds.  Furthermore  we  want  to  show  how  an  admissible  optimal  sensor  schedule  is 
determined  once  the  pair  Ui,U2  is  known. 

We  shall  prove  that 


U,(  77,0)=  inf  ./(»(•))'  >  —  1.2  (3.48) 

'<  (o)— 1 

;>(0)  =  ;t 


where  n  G  H+  satisfies  (7r,  J)  =  1.  An  optimal  schedule  will  be  constructed  as  follows. 
Suppose,  to  fix  ideas  that  i  —  1.  Then  define 

rj*  =  inf  +  Lhiviit).  0)  (3.49) 

where  again  p,(t)  is  the  solution  of  (3.2).  We  write 

p’{t)=Pl{t),  t  G  [0,r;j.  (3.50) 

Next  define 

r2  =  r'<l<r^2^2^’^’=  K*(p2W  +  (3.51) 

In  (3.51),  it  must  be  kept  in  mind  that  p2(t)  represents  the  solution  of  (3.2)  with  j=2,  starting 
at  7"i  with  value  Pi(r1*).  We  then  define 

p[t)=P2(t),  «e[r;,r;j  (3.52) 


Note  that,  unless  t{  =  T, 


r’  >  r’. 


(3.53) 
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otherwise 


=  Jr.(p.K))  +  ^(p.K),  r,-) 

V,(p,(r;),h)  =  tf»0>.W))  +  £MPiK).'i')  (3.54) 

which  is  impossible  since 

-Ki(Pi(n))  >  O^CpiK))  >  0  a.s.  (3.55) 

Similarly  one  proceeds  to  construct  a  sequence  of  r/  <  r2*  <  <  ...  and  the  process  /?*(•). 

We  can  then  prove  the  following. 

Theorem  3.2.  With  the  same  assumptions  as  in  Theorem  S.l  and  in  addition  assuming  that 
(S.ll)  holds;  the  sequence  of  stopping  times  rj* ,  , . . .  defines  an  optimal  admissible  sensor 

schedule. 

Proof:  Considering  (3.10)  as  a  VI  with  obstacle  g2,$i,  we  can  write  from  the  definition  of 

Uii*,  0)  =  ^{f/i(piK),r;) 

+  r  Ci(pi(A))dA}.  (3.56) 

Jo 

This  can  be  established  by  utilizing  the  penalization  (3.23),  along  similar  lines  as  in  [9,  pp. 
578  -  587],  Then 


^{V.(piW),r,-)}  =  E{Ut(p(r;),r;} 

=  E{9{p-(T))x,;=t} 

Substituting  back  in  (3.56)  and  using  the  definition  of  t[  in  (3.49)  we  obtain 

Ui(7T,  0)  =  E  |'^(p*(7’))xti*=t  +  J*1  Ci(p*(A))dA 

+K1{p*{rf))xr-l<T  +  U2{p’{t;),t;)Xt;<t} 

Furthermore,  again  by  employing  penalization  one  can  show  that 

£{£Mp‘(ri),^)}  =  E{U2{p2{rl),Tf)}  =  E{U2(p2{r;),r;) 

+  r  c2{P2{\))d\}. 

Jt1 

This  implies 

E{U2(p2(Tf),T*)Xr;<T}  =  E{U2{p2{T'2),T'2)Xr:<T 

+  Xr;<T  /.  C'2(p2(A))dA}. 


(3.57) 


(3.58) 


(3.59) 
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Next 


■®{^2(P2(f2)»7j)Xr;<r}  —  E{y{p{T))Xr:<Ts;=T} 

+  E{u2(p-(T;),r;)Xr;<T}. 

Substituting  back  in  (3.57)  and  using  the  definition  of  rj  in  (3.51)  we  obtain 

Ui(*,  0)  =  E{<nP-(T))X,;=T  +  Kt(p-(r;))x,;<T 

+K2{p*{t;))x  r*  <r  +  fT'  C,(p*(A))<fA 

«f0 

+  r  C*(p'(A))rfA  +  Cf,(p-K),n;)x,j<T} 

JT; 

Proceeding  in  a  similar  fashion,  and  collecting  results  we  can  write: 

£M*.o)  =  £{*(p'(r))x,;=T 

+  E^i(p'(0)Xr,r<r 


1  =  1 
n— 1 


+  Ex,,v,<r /',+'C..+1(p-(A))dA}, 
«=0  ,/r.* 


where  we  used  the  notation 


(p>; 

)X 

)x 

r*<T 

'  Ku 

if  i 

is 

odd 

,  k2, 

if  i 

is 

even 

'  Cr, 

ifi 

is 

odd 

.  <?*» 

if  i 

is 

even 

Ui, 

ifi 

is 

odd 

ifi 

is 

even. 

However,  observe  that  necessarily  t*  =  T,  for  n  large  enough  (random).  Otherwise 
t*  <  T,\fn ,  on  a  set  fi0  C  H  of  positive  probability.  But  r*  |  r*  <  T  and 


where  (since  (7r,  1)  =  l) 


(see  (2.66))  and 


(p*(7v*),l)  —  (p*(r*),l) 


{p*(T*),l)  =  1+  f  p*STdy 
Jo 


a.s. 

where  f(-)  is  the  process  introduced  by  (2.8).  Therefore  on  n0,  as  n  — ►  oo 

J2Ki(P*(Ti))Xr:<T  - ♦ 

«=1 


(3.60) 

(3.61) 

(3.62) 
one  has 

(3.63) 

(3.64) 

(3.65) 


+oo 


(3.66) 
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and  since  fio  has  positive  probability,  asn-»oo 

E&Mp-Wx^t}  —  00,  (3.67) 

«  =  1 

which  contradicts  (3.19). 

We  can  thus  assert  that 

X  r*  =  T  - ♦  1  0-5.  (3.68) 

In  particular,  it  follows  that  the  sequence  _ ,  defines  an  admissible  schedule  denoted 

by  «*.  The  corresponding  state  solution  of  (2.66),  coincides  with  p*  and  (3.61)  implies 

Ui(n,0)  >  J(u*(-)).  (3.69) 

But  by  standard  arguments,  one  checks  that 

£/i(tt,0)  <  J(u(-)),  Vu(-)  G  Uad  (3.70) 

and  therefore  u'{-)  is  indeed  optimal. 

3.4.  The  main  result 

We  want  now  to  get  rid  of  (3.1l)  and  consider  the  original  functional  in  (3.8).  Let  us 
consider  the  approximation  (3.12)  of  tlb  To  corresponds  a  system  of  QVI. 

e  c(o,T\ Ci) 

u?,u?  >  0 

U?{1 r,r)  -  £/?(*,  T)  =  ¥„(*) 

<  ^(s-t)!/,"^,*)  +  I’  -tjC^TTjdX 

U?{7T,t)  <  $>2{s-t)U?{n,s)  +  j’  $2{\  -  t)C2{n)d\ 

Vs  >  t 

U?(ir,t)  <  KiW  +  Uffat) 


<  K2(7r)  +UP(7Tft). 

From  Theorem  3.2,  we  can  assert  that 

(3.71) 

u?{*,  0)=  inf  ■/>(•)), 

u(0)=t 

i  —  1,2 

(3.72) 

p(0)=7T 


where 

J"(u(-))  -  E{*n(p(u(-),T))+  [T(p(u(-),t),C(u(t)))dt 

■J  o 
oo 


(3.73) 
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Therefore  we  deduce  that 


/>(•))  -  J(n(-))  =  £{*n(p(u(-),T))  -  9(p(v(-),T))} 


and  from  (3.12)  we  deduce 


(3.74) 


l-'XO)  -  •/(«(•))!  <  sff&V'p M'jJ 

U  »  +  IMP  / 


But  using  the  equation  (2.50)  yields  (see  (2.1a)) 


[  f  P(u(') i  T)x{l  +  - jj  ...  -)rfxl  x  — — - — - L3  75) 

V  (i  +  M_)i/2'  J  fp(u(.),T)dxj{3-75) 


p(u(-)>t)IM4 


n  +  llxl 


dx  >  —  E 


1 !  p(u{-),s){x)  W(2!L±M!fe 

J  ’  1  dxi  (n  +  |x|2)2 


4-„..  (  +  IMP)  8 XiXjTl2 

wr  (» + mrw 


2[|i||!(2u  +  ||x||2)i,. 


(n  +  ||x|P)i 


MM  IMP 


+  !M 


"tvenilmly  Pl0ye<*  SUmmati°n  COnVention  over  repeated  indices.  Hence  after  majorizing 


p(u(-),0(M!MP 


.j/*t 


r\  <  [  MM  IMP 

J  “  J  n+\\x\ 


+  rlE{r-^trW^dxlds+^  w 

“  Greek  ICUerS'  '' A’-’  t0  indkate  estimates. 


p(M-)>M(MIMP 


n  +  llxl 


dx  /  ds  ~f- 


,  i  [  rW  ),t)(x) Until-  l 

V  ~VEizp-dxj 


dx}  <  r, 


MMIM II4 ,  ,  i 

n  +  Wdz+n 


<  r, 


Next  consider 


i/u(*)H  Vx+i],  (3.77) 


MMO.O 


(p(u(-),t),TL) 

which  is  the  normalized  conditional  probability,  measure  and  satisfies  Kusner’s  equation 
MMO(M)  =  o{t)(L<p)dt  +  ( a(t)(h<p )  -  a(t)(<p)a(t)(h))  •  (dz  -  a(t)(h)dt) 


(3.78) 
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If  we  apply  (3.78)  with  <p  =  ||x||2  =  X2,  »«  °b‘“" 

<iEMt)(x2)}  =  ®M()(®x2)  -  "W  WM^fe2)  -  <t(t)(x!)<’('*)l} dt 

<  A0(l  +  B{«(1)(X2))- 


Finally 


E{ct(0(x2)>  <A*f  Ax)\\xfdx 


But  the  2nd  term  in  (3.75)  is 


E  ■  °{T)  {x{l  +  MT)[x{1  (l  +  £)''2>) 


< 


E  HT)(x(1  +  -^T2^ 


1/2 


e{Wp(t)  x(i 


<  A1(E{ff(T)(x2)})‘/2  (®  jlWr)  (x(1  “  “*}) 

«  1  /2 

e(||p(T)  (x(1 


(1  +  £)■'* 

1/2 


<  A 


<  A3 


(i  +  £)1/2 

(  \  \ 2> 
E  |p(T)(x2)p(r)  (j^TJt) 


1/2 


One  easily  checks  that 

E{(p(r)(x2))’}  <  A*+(/x(x)||x||2<Jx)  <A 


«®{wo(^)lJ}  *  “M1- 


+  X2 


pW(rr^)  \dt 


n  +  x 


+  A  *E 


But 


hence 


n  +  X" 


< 


y/n 


dE 


■ 

+  ^. 


dt 


(3.79) 


(3.80) 


1/2 


(3.81) 


(3.82) 

(3.83) 
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which  implies 


*(H0(^-,n|se, 


Therefore,  continuing  from  (3.81),  the  2nd  term  in  (3.75)  is  majorized  by  -fa.  Collecting 
results  (from  (3.75),  (3.77),  (3.81),  (3.84))  we  can  assert  that 

|J"(«(0)  -  J(«(-))l  <  (3-85) 

provided  the  initial  distribution  of  p(0),  i.e.  n  satifies 

J  7r(x)\\x\\4dx  <  oo  (3.86) 

The  estimate  in  (3.85)  is  uniform  with  respect  to  n.  Therefore 

ICCOr.O)  -  inf  .  ■/(<.(•))!  <  ~  (3.87) 

u(o)=t  n  / 

p(0)  =  7T 

In  fact  we  can  replace  0  by  any  t  €  [0,  T]  and  consider  the  function 

Ui{*,t)=  inf  J((u(-))  (3.88) 

p(t)-n 

where  J£(u(-))  corresponds  to  a  problem  analogous  to  (2.50),  (2.61)  starting  in  t  instead  of  0. 
Therefore  we  have 

|U,fl(7r,t)-tf,(7r,t)|<^7j  (3.89) 

We  have  however  to  be  careful  to  the  fact  that  the  constant  in  (3.89)  depends  on  a  bound  on 
/  7r(x)||x|| 4dx.  More  precisely  we  have  proved  that 

I U?{ir,t)  -  Ui(7T,t)\  <  ~(1  +  j  ir(x)\\x\\4dx)  (3.90) 

where  A'  this  time  does  not  depend  on  n  (assuming  that  n  is  a  probability).  It  follows  that 

U?{n,t)  — +  Ui(n,t)  in  <7(0,  T;  Cx).  (3.91) 

Taking  the  limit  in  (3.71),  we  obtain  that  UitU2  is  a  solution  of  (3.10)  and  moreover 


Ui{v, 0)  =  inf  .  J («(•)) 

u(0)=t 

p(0)=7T 


(3.92) 


However  by  a  probabilistic  argument  already  used  in  section  3.3,  any  solution  of  (3.10)  is 
smalller  than  the  right  hand  side  of  (3.92).  This  completes  the  proof  of  Theorem  3.1,  and 
also  provides  the  same  statement  as  in  Theorem  3.2,  without  the  assumption  (3.11)  and  for 
our  original  given  by  (3.8). 
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